ADF Data Cube to HDF5 Mapping Ontology v1.5.3 RF

The Allotrope Data Format (ADF) [[!ADF]] consists of several APIs, taxonomies and ontologies. This document describes the Allotrope Data Format Data Cube to HDF5 Mapping Ontology (ADF-DCO-HDF). While the ADF Data Cube Ontology [[!ADF-DCO]] provides the business description of n-dimensional data cubes, ADF-DCO-HDF provides the vocabulary and structure for mapping these data cubes to HDF5 datasets [[!HDF5]]. The ADF-DCO itself is based on the RDF Data Cube Vocabulary [[!QB]]. However, while in QB and correspondingly ADF-DCO, data is expressed in form of triples, ADF in general allows data to be stored also natively in HDF5 file format which is more efficient, when dealing with large volumes of numerical data. The main entity in ADF-DCO-HDF is the HDF Mapping which consists of an index structure definition, one or more scale mappings and one mapping structure definition.

Disclaimer

THESE MATERIALS ARE PROVIDED "AS IS" AND ALLOTROPE EXPRESSLY DISCLAIMS ALL WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, INCLUDING, WITHOUT LIMITATION, THE WARRANTIES OF NON-INFRINGEMENT, TITLE, MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE.

This document is part of a set of specifications on the Allotrope Data Format (ADF)[[!ADF]].


Introduction

The Allotrope Data Format (ADF) [[!ADF]] defines an interface for storing scientific observations from analytical chemistry. It is intended for long-term stability of archived analytical data and fast real-time access to it. The ADF Data Cube API (ADF-DC) defines an interface for storing raw analytical data. The meta data of data cubes is specified in terms of ADF Data Cube Ontology (ADF-DCO) [[!ADF-DCO]] which is based on the RDF Data Cube Vocabulary [[!QB]]. ADF-DCO-HDF provides classes and properties to define the mapping between the abstract data cubes defined by the terms of the data cube ontology to their concrete HDF5 representations in the ADF file. The structure and metadata of HDF5 objects are described by an HDF5 ontology, which is based on the HDF5 specifications [[!HDF5]].

This document is structured as follows: First, the role of the ADF Data Cube API within the high-level structure of ADF [[!ADF]] API stack is presented. Then, the requirements for the ADF Data Cube to HDF5 Mapping Ontology are described, and an overview of the structure of ADF-DCO-HDF and the relations to ADF-DCO and the HDF5 are summarized.

ADF-DCO-HDF will be published under http://purl.allotrope.org/ontologies/datacube-hdf-map

Document Conventions

Naming Conventions

The IRI of an entity has two parts: the namespace and the local identifier. Within one RDF document the namespace might be associated by a shorter prefix. For instance the namespace IRI http://www.w3.org/2002/07/owl# is commonly associated with the prefix owl: and one can write owl:Class instead of the full IRI http://www.w3.org/2002/07/owl#Class.

Within the biomedical domain the local identifier is often an alphanumeric ID which is not human readable. The Allotrope Foundation Taxonomies follow this approach, e.g. a process is represented as af-p:AFP_0001617. To enhance readability within this document, the preferred label from the ontology or taxonomy is used for the corresponding entity. I.e., instead of af-p:AFP_0001617 the corresponding entity is named as af-p:process. If the namespace is clear by the context the prefix MAY be omitted and the entity is named simply process. If the label contains spaces, the entity MAY be surrounded by Guillemets to avoid ambiguities, e.g. «af-p:experimental method».

Number Formatting

Within this document, decimal numbers will use a dot "." as the decimal mark.

Namespaces

Within this specification, the following namespace prefix bindings are used:

Prefix Namespace
owl:http://www.w3.org/2002/07/owl#
rdf:http://www.w3.org/1999/02/22-rdf-syntax-ns#
rdfs:http://www.w3.org/2000/01/rdf-schema#
xsd:http://www.w3.org/2001/XMLSchema#
skos:http://www.w3.org/2004/02/skos/core#
dct:http://purl.org/dc/terms/
qudt:http://qudt.org/schema/qudt#
qudt-unit:http://qudt.org/vocab/unit#
qudt-quantity:http://qudt.org/vocab/quantity#
qb:http://purl.org/linked-data/cube#
af-c:http://purl.allotrope.org/ontologies/common#
af-m:http://purl.allotrope.org/ontologies/material#
af-e:http://purl.allotrope.org/ontologies/equipment#
af-p:http://purl.allotrope.org/ontologies/process#
af-r:http://purl.allotrope.org/ontologies/result#
af-x:http://purl.allotrope.org/ontologies/property#
adf-dp:http://purl.allotrope.org/ontologies/datapackage#
adf-dc:http://purl.allotrope.org/ontologies/datacube#
adf-dc-hdf:http://purl.allotrope.org/ontologies/datacube-to-hdf5-map#
hdf:http://purl.allotrope.org/ontologies/hdf5/1.8#
afs-qudt:http://purl.allotrope.org/shapes/qudt#
afs-dc:http://purl.allotrope.org/shapes/datacube#
afs-dc-hdf:http://purl.allotrope.org/shapes/hdf-dc#
afs-hdf:http://purl.allotrope.org/shapes/hdf#
ex:http://example.com/ns#

Indication of Requirement Levels

Within this document the definitions of MUST, SHOULD and MAY are used as defined in [[!rfc2119]].

ADF High-Level Structure

The following figure illustrates the high-level structure of the Allotrope Data Format (ADF) API stack:

The high-level structure of the Allotrope Data Format (ADF) API stack.

This document focuses on the ADF Data Cube to HDF5 Mapping Ontology, which is used by the ADF Data Cube API [[ADF-DC]] highlighted in the figure above. The mapping ontology provides the bridge between the Data Cube API and the physical storage in HDF5.

Requirements

The ADF Data Cube to HDF5 Mapping Ontology (ADF-DCO-HDF) provides a data model for the mapping of data cubes to HDF5 datasets. Thus, the key requirements regarding of the ADF Data Cube Ontology are the following: ADF-DCO-HDF MUST provide data structures for

The following aspects MUST be considered when designing the mapping ontology to ensure usability, storage efficiency and performance:

ADF Data Cube to HDF5 Mapping Ontology High-Level Structure

The following figure illustrates the high-level structure of the ADF Data Cube (ADF-DC) API with the ADF Data Cube Ontology and its components.

ADF Data Cube to HDF5 Mapping Ontology is the bridge between ADF DCO and HDF5.

ADF-DCO-HDF imports the ADF Data Cube Ontology (ADF-DCO) and the HDF5 ontology and thus defines a mapping between the business perspective and the physical representation of n-dimensional data in HDF5.

The ADF Data Cube Ontology

The ADF Data Cube ontology (ADF-DCO) extends the RDF Data Cube Vocabulary (QB) by scales, data selections, order functions and complex data types. Details on ADF-DCO can be found in the corresponding specification. The most important classes and relations are illustrated in the figure below. Details can be found in [[!ADF-DCO]].

The HDF5 Ontology

The HDF5 ontology is based on the HDF5 specification. Sources for development were the DDL file and the HDF5 User's Guide. The role of the HDF5 ontology is to provide the structure and vocabulary for description of the physical representation of data stored in HDF5.

High-Level Class Relations

The following figure illustrates the relation between high-level classes of the mapping ontology with classes from the ADF Data Cube Ontology and the most important ones of the HDF5 Ontology.

High-level relations between classes of the ADF Data Cube Ontology (ADF-DCO), classes of the RDF Data Cube Vocabulary (QB), the Mapping ADF-DCO-HDF mapping ontology and the HDF5 ontology.

ADF-DCO extends QB, HDF5 ontology provides the vocabulary to describe HDF5 data representations. ADF-DCO-HDF provides a bridge between the business view (expressed in QB and ADF-DCO terms) and the physical representation in HDF5 datasets. The QB classes qb:DataSet, qb:DataStructureDefinition and qb:ComponentSpecification remain central in ADF-DCO. Extension for selections and the mappings are defined in a parallel structure. A HDF5 mapping consists of an IndexStructureDefinition, one or more ScaleMappings and one MappingStructureDefinition, which in turn defines ComponentMappings. The details of the mapping are described in the next section.

ADF Data Cube to HDF5 Mapping Ontology - Concept Details

A mapping defines the mapping between the data structure definition of an RDF data cube and corresponding HDF5 datasets which contain the actual observations (or data points). A mapping consists of three essential parts:

The three parts of the HDF5 mapping are explained in detail in the following sections.

An HDF5 mapping consists of an index structure definition, scale mappings and a mapping structure definition.

Index Structure Definition

In ADF-DCO, measures are components which are dependent of dimension components. Thus, the dimension components are used to specify an index for measurements (or measurement items). The index structure definition (ISD) is part of an HDF5 mapping and defines a set of indexes for the different dimension components of a data structure definition.

Index structure definition with index and dimension component.
Index

An index is part of index structure definition and related to at least one dimension component of a data structure definition. All indexes of the ISD form a partition of the set of dimension components of the data structure definition. The values of the different dimension components are indexed according to their order number. The exact definition of the index MUST be specified by the framework. The order number of the index itself MAY be used by the framework to define the overall index structure on storage level.

If a data set has multiple dimension components, there are different ways to realize an ISD. The indexes of the ISD MUST define a partition of the set of dimension components. That is, for each dimension component there is exactly one index defined, but one index can cover multiple dimension components. Thus, there are two main strategies:

The next example illustrates the usage of an ISD for a cube with three dimensions - all with separate indices. 

ex:ISD a adf-dc-hdf:IndexStructureDefinition ;
adf-dc-hdf:index [
    a adf-dc-hdf:Index ;
    adf-dc-hdf:indexOn ex:dimension1 ;
    adf-dc:order "1" ;
] ;
adf-dc-hdf:index [
    a adf-dc-hdf:Index ;
    adf-dc-hdf:indexOn ex:dimension2 ;
    adf-dc:order "2" ;
] ;
adf-dc-hdf:index [
    a adf-dc-hdf:Index ;
    adf-dc-hdf:indexOn ex:dimension3 ;
    adf-dc:order "3" ;
] .
The instance of an adf-dc-hdf:Index defines which dimensions are indexed together. The actual representation of the index MUST be implemented by the framework. For instance a cross product indexing MAY be defined for indexing multiple dimensions.

Scale Mapping

A scale mapping is part of a mapping and maps exactly one scale of a corresponding data structure definition from technical indices to scale values.

A scale mapping defines the mapping of technical index for scale values.
Identity Scale Mapping

An identity scale mapping is a scale mapping where the scale value and the technical index are identical.

Explicit Scale Mapping

An explicit scale mapping is a scale mapping which maps a scale by specifying a dimension mapping which provides an explicit representation of the mapping through some HDF5 dataset.

Function Scale Mapping

A function scale mapping is a scale mapping which is realized by an index function. Function scale mappings allow efficient access to scale values and also save storage space. Index functions work on numbers. The consequence is that the dimension's adf-dc:componentDatatype MUST be either a numeric datatype or if it is a complex datatype (shape), it MUST have an identifying property path stated by adf-dc:propertyPath, that points to a numeric datatype. In all other cases an explicit scale mapping MUST be used. 

                     ex:DurationDimension a adf-dc:Dimension, adf-dc:IntervalScale;
                        qb:dimension «af-x:time» ;
                        adf-dc:componentDatatype ex:DurationType ;
                        .
                     ex:DurationType a sh:Shape;
                        sh:property [
                           sh:predicate qudt:numericValue;
                           sh:datatype xsd:double;
                           sh:minCount 1;
                           sh:maxCount 1;
                        ] , [
                           sh:predicate qudt:unit;
                           sh:minCount 1;
                           sh:maxCount 1;
                           sh:hasValue qudt-unit:SecondTime; # units are constant!
                        ];
                        .
                     ex:DurationDimensionScaleMapping a adf-dc-hdf:FunctionScaleMapping;
                        adf-dc:propertyPath "qudt:numericValue"; # property path to the number of seconds, a primitive value
                        adf-dc-hdf:indexFunction [
                           a adf-dc-hdf:LinearFunction; # f(i) = 0.1 × i.
                           adf-dc-hdf:parameter1 0E0;
                           adf-dc-hdf:parameter2 1E-1;
                        ];
                        .

A later version will allow the use of quantity values for indexing. The quantity is then equivalent to the value based on a normalized unit using the conversion factors and offsets defined for the unit. For this to work, the implementation MUST have access to the unit ontology. Currently this function is not implemented, so function scales work for quantity values, only if the shape states that the unit is constant.

The framework MUST provide at least the functions described in the next subsection:

Index Functions

An index function is a injective map from natural numbers to real numbers. ADF-DCO defines the following four index functions: LinearFunction, BinaryLogarithm, CommonLogarithm and NaturalLogarithm. All of these functions can be specified by its parameters as shown in the next figure. The framework MUST support all four index functions.

The four different index functions which MAY be used to define function scale mappings.

The functions are defined as follows:

LinearFunction
A linear scale with the formula f(i) = p1 + p2 × i
BinaryLogarithm
A binary logarithm scale with the formula f(i) = p1 × 2 p2×i
CommonLogarithm
A common logarithm scale with the formula f(i) = p1 × 10 p2×i
NaturalLogarithm
A natural logarithm scale with the formula f(i) = p1 × e p2×i
with
i: the index - an integer starting from 0
p1: the first function parameter adf-dc-hdf:parameter1
p2: the second function parameter adf-dc-hdf:parameter2

Mapping Structure Definition

A mapping structure definition (MSD) describes how data types and data set structures defined in RDF with ADF-DCO are mapped to HDF5 datatypes and data sets. It consists of component mappings which define primitive or property mappings and refer to HDF5 datatype mappings.

A component mapping is part of a mapping structure definition and maps a component specification (dimension or measure) to one or more HDF5 datasets. A component mapping has two subclasses DimensionMapping and MeasureMapping which are used to map dimension and measure components respectively as shown in the next figure:

A component mapping is part of a mapping structure definition and defines primitive or property mappings, referring to HDF5 datatype mappings.
The property adf-dc-hdf:mapsComponent is used for reference of the component specification at component mapping level. Depending on the associated component data type, different mappings are used: RDF data types include XSD data types such as xsd:integer or xsd:double, rdf:Resource etc. and also complex data types defined by shapes (which are possibly nested). The property adf-dc-hdf:representedBy is used for reference of the HDF5 dataset for primitive or property mappings. In the following subsections component mappings are described in detail and illustrated by examples.

RDF to HDF5 Mapping

A RDF to HDF5 Mapping is part of a Component mapping and describes a mapping from an RDF type to an HDF5 type for a component data type or an attribute thereof. It is realized either by a adf-dc-hdf:PrimitiveMapping or a adf-dc-hdf:PropertyMapping.

HDF5 Datatype Mapping

The basic element of a primitive or property mapping is the adf-dc-hdf:HDFDataTypeMapping, which defines the low level mapping of RDF data types such as xsd:integer or xsd:double to HDF5 datatypes (e.g. H5T_IEEE_F64BE). An example of an HDF5 datatype mapping is given in the following: 

# primitive data type mapping for xsd:double
ex:primitiveTypeMapDouble a adf-dc-hdf:HDFDataTypeMapping;
    adf-dc-hdf:rdfType xsd:double;
    adf-dc-hdf:hdfType hdf:H5T_IEEE_F64BE .

These adf-dc-hdf:HDFDataTypeMappings MAY be used in more complex settings as shown in the examples below.

Dictionaries

HDF5 is specifically designed to store numerical data. It is however not efficient to store strings in HDF5 datasets. ADF-DCO allows strings or IRIs to be used for component data types. These values are stored in a dictionary.

The dictionary functionality MUST be supported by the framework.

Primitive Mapping

A primitive mapping (adf-dc-hdf:PrimitiveMapping) is part of a component mapping and defines an HDF5 datatype mapping for a primitive RDF data type and an HDF5 dataset. The following example describes a result set of simple temperature measurements and a corresponding mapping to HDF5 datasets. 

# example observation
ex:obs01
    a qb:Observation;
    qb:dataSet ex:TempatureMeasurementResult;
    «af-x:index» 1;
    «af-x:temperature» 34.8 .      # primitive value

# The data set with data structure definition
ex:TempatureMeasurementResult
    a qb:DataSet;
    rdfs:label "temperature measurement result 123" ;
    qb:structure ex:TempResultStructure .

ex:TempResultStructure
    a qb:DataStructureDefinition;
    qb:component ex:IndexDimension;
    qb:component ex:PrimitiveTemperatureMeasure .

ex:IndexDimension
    a qb:ComponentSpecification;
    qb:dimension «af-x:index»;
    qb:order 1;
    af-dc:hasComponentDataType xsd:integer .

ex:PrimitiveTemperatureMeasure
    a qb:ComponentSpecification;
    qb:measure «af-x:temperature»;
    af-dc:hasComponentDataType xsd:double . # a primitive component data type
The corresponding HDF5 datasets are specified as follows: 
ex:HDFDataSpace
    a hdf:DataSpace ;
    hdf:dimension ex:HDFIndexDimension .

ex:HDFIndexDimension
    a hdf:DataSpaceDimension ;
    hdf:index "0"^^xsd:int ;
    hdf:currentSize "5"^^xsd:long ;
    hdf:maximumSize "5"^^xsd:long .

ex:HDFPrimitiveMeasure
    a hdf:Dataset ;
    hdf:datatype hdf:H5T_STD_I16BE ;
    hdf:fillValueStatus hdf:H5D_FILL_VALUE_DEFAULT ;
    hdf:dataspace ex:HDFDataSpace .
The mapping structure definition has two parts: One component mapping for the index dimension and one for the temperature measure. 
ex:MappingStructureDefinition a adf-dc-hdf:MappingStructureDefinition ;
# first part: mapping of the index dimension
adf-dc-hdf:componentMapping [
    a adf-dc-hdf:DimensionMapping;
    adf-dc-hdf:mapsComponent ex:IndexDimension ;
    adf-dc-hdf:dataSpaceDimension ex:HDFIndexDimension ;
    adf-dc-hdf:rdfMapping [
        a adf-dc-hdf:PrimitiveMapping ;
        adf-dc-hdf:hdfDataTypeMapping [
            adf-dc-hdf:rdfType xsd:integer;
            adf-dc-hdf:hdfType hdf:H5T_STD_U32BE ;
        ] ;
    ] ;
];
# second part: mapping of the temperature measure
adf-dc-hdf:componentMapping [
    a adf-dc-hdf:MeasureMapping;
    adf-dc-hdf:mapsComponent ex:PrimitiveTemperatureMeasure ;
    adf-dc-hdf:rdfMapping [
        a adf-dc-hdf:PrimitiveMapping ;
        adf-dc-hdf:hdfDataTypeMapping [
            a adf-dc-hdf:HDFDataTypeMapping ;
            adf-dc-hdf:rdfType xsd:double;
            adf-dc-hdf:hdfType hdf:H5T_IEEE_F64BE;
        ] ;
    ] ;
].

Property Mappings for Complex Data Types

Measures (and also dimensions) MAY have complex component data types. In this case the component mapping MUST define HDF5 datatype mappings for all primitive values of the complex data type. This is shown in the following example where the measure component represents a weight measurement which consists of a numeric value, a standard uncertainty and a unit. 

# example observation
ex:obs01 a qb:Observation;
qb:dataSet ex:WeighingResult;
«af-x:index» 1;
«af-x:net weight» [          # complex value
 qudt:numericValue "100.0"^^xsd:double;
 qudt:standardUncertainty "0.3"^^xsd:double;
 qudt:unit qudt-unit:Gram
] .

# The data set with data structure definition
ex:WeighingResult   a qb:DataSet;
                    rdfs:label "weighing result 123" ;
                    qb:structure ex:MassResultStructure .
                    ex:MassResultStructure a qb:DataStructureDefinition;
                    qb:component ex:IndexDimension;
                    qb:component ex:ComplexMassMeasure .
ex:IndexDimension   a   adf-dc:Dimension ,
                    adf-dc:OrdinalScale ;
                    qb:dimension «af-x:index»;
                    qb:order 1;
                    af-dc:hasComponentDataType xsd:integer .
ex:ComplexMassMeasure a adf-dc:Measure;
                    qb:measure «af-x:net weight»;
                    af-dc:hasComponentDataType ex:MassDataType . # references a complex data type
The complex data type for the mass measurement is represented by a shape: 
ex:MassDataType a sh:Shape ;
sh:property [
  sh:predicate qudt:numericValue;
  sh:minCount 1;
  sh:maxCount 1;
  sh:nodeKind sh:Literal ;
  sh:datatype xsd:double ;
];
sh:property [
  sh:predicate qudt:standardUncertainty;
  sh:minCount 1;
  sh:maxCount 1;
  sh:nodeKind sh:Literal ;
  sh:datatype xsd:double ;
];
sh:property [
  sh:predicate qudt:unit;
  sh:minCount 1;
  sh:maxCount 1;
  sh:nodeKind sh:IRI;
  sh:class qudt:Unit;
];
.
For each primitive data type of the complex data type a separate HDF5 dataset is defined. 
ex:HDFDataSpace a hdf:DataSpace ;
hdf:dimension ex:HDFIndexDimension .

ex:HDFIndexDimension a hdf:DataSpaceDimension ;
hdf:index "0"^^xsd:int ;
hdf:currentSize "50"^^xsd:long ;
hdf:maximumSize "50"^^xsd:long .

ex:HDFComplexMeasureNumericValue a hdf:Dataset ;
hdf:datatype hdf:H5T_IEEE_F64BE ;
hdf:fillValueStatus hdf:H5D_FILL_VALUE_DEFAULT ;
hdf:dataspace ex:HDFDataSpace .

ex:HDFComplexMeasureStandardUncertainty a hdf:Dataset ;
hdf:datatype hdf:H5T_IEEE_F64BE ;
hdf:fillValueStatus hdf:H5D_FILL_VALUE_DEFAULT ;
hdf:dataspace ex:HDFDataSpace .

ex:HDFComplexMeasureUnit a hdf:Dataset ;
hdf:datatype hdf:H5T_STD_I64BE ;            # URIs are represented as integers
hdf:fillValueStatus hdf:H5D_FILL_VALUE_DEFAULT ;
hdf:dataspace ex:HDFDataSpace .
The mappings of the different primitive data types are defined as follows. Firstly, a default mapping for xsd:double values is defined: 
ex:DefaultDoubleMapping a adf-dc-hdf:HDFDataTypeMapping ;
adf-dc-hdf:rdfType xsd:double;
adf-dc-hdf:hdfType hdf:H5T_IEEE_F64BE . 		    # HDF5 type is a predefined or custom HDF5 datatype
Here, the RDF type is either one of the standard rdf data types, rdf:Resource (any IRI) or sh:Shape. Then the primitive data types are mapped (shown for measure components): 
ex:MappingStructureDefinition a adf-dc-hdf:MappingStructureDefinition ;
adf-dc-hdf:componentMapping [
    a adf-dc-hdf:MeasureMapping;
    adf-dc-hdf:mapsComponent ex:ComplexMassMeasure ;
    adf-dc-hdf:rdfMapping [
        a adf-dc-hdf:PropertyMapping ;
        adf-dc-hdf:hdfDataTypeMapping ex:DefaultDoubleMapping ;
        adf-dc:propertyPath "<http://qudt.org/schema/qudt#numericValue>" ;
        adf-dc-hdf:hdfDataset ex:HDFComplexMeasureNumericValue ;
    ] ;
    adf-dc-hdf:rdfMapping [
        a adf-dc-hdf:PropertyMapping ;
        adf-dc-hdf:hdfDataTypeMapping ex:DefaultDoubleMapping ;
        adf-dc:propertyPath "<http://qudt.org/schema/qudt#standardUncertainty>" ;
        adf-dc-hdf:hdfDataset ex:HDFComplexMeasureStandardUncertainty ;
    ] ;
    adf-dc-hdf:rdfMapping [
        a adf-dc-hdf:PropertyMapping ;
        adf-dc-hdf:hdfDataTypeMapping [
            a adf-dc-hdf:HDFDataTypeMapping ;
            adf-dc-hdf:rdfType rdf:Resource ;
            adf-dc-hdf:hdfType hdf:H5T_STD_I64BE ;
            adf-dc-hdf:usingDictionary ex:dict123 ; # strings and URIs are mapped to integers in HDF5 using a dictionary
        ]
        adf-dc:propertyPath "<http://qudt.org/vocab/unit#unit>" ;
        adf-dc-hdf:hdfDataset ex:HDFComplexMeasureUnit ;
    ] ;
].
ex:dict123 a hdf-ext:Dictionary .
Standard HDF5 Datatype Mappings for RDF Data Types

To ease the definition of mappings, ADF-DCO has the following standard mappings for all RDF data types such as xsd:integer:

RECOMMENDED standard HDF5 datatype mappings for RDF data types.
RDFHDFHDFDataTypeMapping
xsd:doublehdf:H5T_IEEE_F64BEadf-dc-hdf:standardDoubleMapping
xsd:floathdf:H5T_IEEE_F32BEadf-dc-hdf:standardFloatMapping
xsd:integerhdf:H5T_STD_I64BEadf-dc-hdf:standardIntegerMapping
xsd:negativeIntegerhdf:H5T_STD_I64BEadf-dc-hdf:standardIntegerMapping
xsd:positiveIntegerhdf:H5T_STD_I64BEadf-dc-hdf:standardIntegerMapping
xsd:nonNegativeIntegerhdf:H5T_STD_I64BEadf-dc-hdf:standardIntegerMapping
xsd:nonPositiveIntegerhdf:H5T_STD_I64BEadf-dc-hdf:standardIntegerMapping
xsd:longhdf:H5T_STD_I64BEadf-dc-hdf:standardLongMapping
xsd:unsignedLonghdf:H5T_STD_I64BEadf-dc-hdf:standardUnsignedLongMapping
xsd:inthdf:H5T_STD_I32BEadf-dc-hdf:standardIntMapping
xsd:unsignedInthdf:H5T_STD_U32BEadf-dc-hdf:standardUnsignedIntMapping
xsd:shorthdf:H5T_STD_I16BEadf-dc-hdf:standardShortMapping
xsd:unsignedShorthdf:H5T_STD_U16BEadf-dc-hdf:standardUnsignedShortMapping
xsd:bytehdf:H5T_STD_I8BEadf-dc-hdf:standardByteMapping
xsd:unsignedBytehdf:H5T_STD_U8BEadf-dc-hdf:standardByteMapping
xsd:stringhdf:H5T_STD_I32BEadf-dc-hdf:standardStringMapping
xsd:anyURIhdf:H5T_STD_I32BEadf-dc-hdf:standardStringMapping
rdf:Resourcehdf:H5T_STD_I32BEadf-dc-hdf:standardResourceMapping

The data types xsd:string (IRIs) and rdf:Resource are stored as 32bit integers that are keys into dictionaries.

  • xsd:decimal has no mapping.
  • xsd:boolean has no mapping.
  • xsd:duration has no mapping.
  • xsd:date and xsd:dateTime have no mappings and are treated as time integers (ms since 1.1.1970)
  • xsd:integer, xsd:negativeInteger, xsd:positiveInteger, xsd:nonNegativeInteger, xsd:nonPositiveInteger and xsd:unsignedLong are only supported in the range of signed long 64bit integers.
    • Mapping Multiple Measures to HDF

    A data cube contains one or many adf-dc:Measures which need to be mapped to HDF5 datasets. In case of multiple measures there are three approaches to map them to HDF5 datasets:

    • One HDF5 dataset per measure.
    • One HDF5 compound data set with one member per measure.
    • One HDF5 dataset that has an artificial dimension with a nominal scale having as scale values the names of the measures. This approach is only applicable if all measures have the same data type.
    The use of of HDF5 compound data sets is discouraged, because search becomes less efficient.
    If the measure component data type is complex, by default, multiple HDF5 datasets will be used for their representation. The following example illustrates how multiple measures MAY be mapped to one HDF5 dataset. The example observation has a tare weight and a net weight with value and error. All values are of type xsd:double: 

    ex:obs01 a qb:Observation;
   qb:dataSet ex:WeighingResult;
   «af-x:index» 1;
   «af-x:tare weight» '120.912'^^xsd:double     # primitive value
   «af-x:net weight» [                          # complex value
     qudt:numericValue '100.014'^^xsd:double ;
     qudt:standardUncertainty '0.326'^^xsd:double ;
  ] .

    When multiple measures are mapped to the same HDF5 dataset, an HDF5 field index and dimension index (both of type xsd:long) MUST be specified. The mapping for the measures looks as follows. 

    ex:MappingStructureDefinition a adf-dc-hdf:MappingStructureDefinition ;
    adf-dc-hdf:componentMapping [
        a adf-dc-hdf:MeasureMapping;
        adf-dc-hdf:mapsComponent ex:SimpleTareWeightMeasure ;
        adf-dc-hdf:rdfType xsd:double ;
        adf-dc-hdf:rdfMapping [
            a adf-dc-hdf:PrimitiveMapping ;
            adf-dc-hdf:hdfDataTypeMapping ex:DefaultDoubleMapping ;
            adf-dc-hdf:hdfFieldIndex "0"^^xsd:long ;             # simple tare weight -> field dimension = 0
            adf-dc-hdf:hdfDimensionIndex "0"^^xsd:long           # from dimension «af-x:index»
            adf-dc-hdf:hdfDataset ex:HDFMultiMeasureDataSet ;    # multi measure data set
        ] ;
    adf-dc-hdf:componentMapping [
        a adf-dc-hdf:MeasureMapping;
        adf-dc-hdf:mapsComponent ex:ComplexNetWeightMassMeasure ;
        adf-dc-hdf:rdfMapping [
            a adf-dc-hdf:PropertyMapping ;
            adf-dc-hdf:hdfDataTypeMapping ex:DefaultDoubleMapping ;
            adf-dc:propertyPath "qudt:numericValue";
            adf-dc-hdf:hdfFieldIndex "1"^^xsd:long ;             # complex net weight.numeric value -> field dimension = 1
            adf-dc-hdf:hdfDimensionIndex "0"^^xsd:long           # the same dimension index
            adf-dc-hdf:hdfDataset ex:HDFMultiMeasureDataSet ;    # multi measure data set
        ] ;
        adf-dc-hdf:rdfMapping [
            a adf-dc-hdf:PropertyMapping ;
            adf-dc-hdf:hdfDataTypeMapping ex:DefaultDoubleMapping ;
            adf-dc:propertyPath "qudt:standardUncertainty";
            adf-dc-hdf:hdfFieldIndex "2"^^xsd:long ;             # complex net weight.standard uncertainty -> field dimension = 2
            adf-dc-hdf:hdfDimensionIndex "0"^^xsd:long           # the same dimension index
            adf-dc-hdf:hdfDataset ex:HDFMultiMeasureDataSet ;    # multi measure data set
        ] ;
    ].
    Mapping of all values/properties of the shape is mandatory.

    Change History

    Version Release Date Remarks
    0.4.0 2015-06-18
    • Initial Working Draft version
    1.0.0 RC 2015-09-17
    • Complete Revision after split of ADF Data Cube ontology and ADF Data Cube to HDF5 Mapping Ontology
    1.0.0 2015-09-29
    • Incorporated feedback of reviews
    • Updated versions, dates and document status
    1.1.0 RC 2016-03-11
    • Updated versions, dates and document status
    • Added section on number formatting to document conventions
    • Updated Fig. 1 and Fig. 3, resized Fig. 8
    1.1.0 RF 2016-03-31
    • Updated versions, dates and document status
    1.1.5 2016-05-13
    • Updated versions and dates
    1.2.0 Preview 2016-09-23
    • Updated versions and dates
    1.2.0 RC 2016-12-07
    • Updated versions and dates
    1.3.0 Preview 2017-03-31
    • Updated versions and dates
    1.3.0 RF 2017-06-30
    • Updated versions and dates
    • Adaptations to new business model
    • Updated section 5.3.6
    • Minor edits
    1.4.3 RC 2018-10-11
    • Updated versions and dates
    1.4.5 RF 2018-12-17
    • Updated versions and dates
    1.5.0 RC 2019-12-12
    • Updated versions and dates
    1.5.0 RF 2020-03-03
    • Updated HDF5 reference link
    1.5.3 RF 2020-11-30
    • Updated broken reference links
    • Updated PURL and DOCS server links to relative links
    • Reformat the document header